CLIP Driven Few-shot Panoptic Segmentation

This paper presents CLIP Driven Few-shot Panoptic Segmentation (CLIP-FPS), a novel few-shot panoptic segmentation model that leverages the knowledge of Contrastive Language-Image Pre-training (CLIP) model. The proposed method builds upon center indexing attention mechanism to facilitate transfer, which entails representing objects in an image as centers along with their pixel offsets. comprises decoder responsible for generating object center-offset groups and self-attention module tasked producing feature map. Subsequently, index map acquire corresponding embeddings, paving way matrix multiplication SoftMax operation text embedding matching computation final masks. Quantitative evaluation on datasets such COCO Cityscapes shows our outperforms existing techniques terms Quality (PQ) metrics.